https://www.boundaryml.com logo
🟡 BAML Atomic List-Item Streaming Support Inquiry
# questions
d
Does BAML support atomic list‑item streaming? I’m streaming a function that returns a list of union items (e.g., tool calls and messages). What I want: each list element should only appear once it’s fully complete—no partial updates to that element after it first shows up. Example shape: ⁨`type OutputItem = ToolCall | Message`⁩ ⁨`function Run(...) -> OutputItem[]`⁩ In practice, I still see partial updates: an item appears with placeholder/empty fields and then updates later as more tokens arrive. I’ve tried using ⁨`@@stream.done`⁩ on the output classes and also wrapping the union in a dedicated class with ⁨`@@stream.done`⁩, but list elements still update. Is “atomic list item” streaming supported? If yes, what’s the correct way to achieve it?
v
Does (T @stream.done)[] not work here?
If not we’ll def fix the bug!
d
Not in every test so far 🙁
With this code, it doesn't actually stream, it's the same as calling get_final_response: ⁨⁨``` class ToolCall { name string @stream.done @stream.not_null parameters string @stream.done } class ExampleMessage { role string @stream.done @stream.not_null content string @stream.done } type OutputItem = ToolCall | ExampleMessage class CompleteItem { item OutputItem @stream.done @stream.not_null @@stream.done } function Run(input: string) -> CompleteItem[] { client "Default" prompt #" Process the following input and return a list of tool calls and messages. {{ input }} {{ ctx.output_format }} "# } ```⁩⁩
v
That’s def a bug. Let me try and repro and hopefully we can fix this in either this release or next!
d
With this code, it does stream, but I get updates where I shouldn't: ⁨``` class ToolCall { name string parameters string @stream.done } class ExampleMessage { role string content string @stream.done } type OutputItem = ToolCall | ExampleMessage function Run(input: string) -> (OutputItem @stream.done)[] { client "Default" prompt #" Process the following input and return a list of tool calls and messages. {{ input }} {{ ctx.output_format }} "# } ```⁩
v
Ok perfect. Thanks for the easy repro
d
Let me know if it was a bug or if I'm messing up 🙂 first time using streaming. This would also be an interesting case for the docs. It's not obvious that you should be able to do this with list items 🙂
so def a bug on our end for how ⁨`(T @stream.done)[]`⁩ was working!
d
Weird, I'm on BAML 0.218.1 and trying this: ⁨⁨```class ToolCall { name string parameters string @@stream.done } class ExampleMessage { role string content string @@stream.done } type OutputItem = ToolCall | ExampleMessage function Run(input: string) -> (OutputItem @stream.done)[] { client "Default" prompt #" Process the following input and return a list of tool calls and messages. {{ input }} {{ ctx.output_format }} "# }
Copy code
⁩⁩

And this Python code:
⁨⁨
from __future__ import annotations from conference_concierge.baml_client.sync_client import b def main() -> None: stream = b.stream.Run( input=""" call 3 tools: - get_weather - add_reminder - reply_to_user and tell the user "hello world" """.strip() ) for chunk in stream: print("--------") for item in chunk: print(item) print("-------- Done --------") done = stream.get_final_response() print(done) if __name__ == "__main__": main()
Copy code
⁩⁩

And the result is:
⁨⁨
-------- name='get_weather' parameters=None -------- name='get_weather' parameters='{}' name='add_reminder' parameters=None -------- name='get_weather' parameters='{}' name='add_reminder' parameters='{}' name='reply_to_user' parameters=None -------- name='get_weather' parameters='{}' name='add_reminder' parameters='{}' name='reply_to_user' parameters='{}' role='assistant' content='hello world' -------- name='get_weather' parameters='{}' name='add_reminder' parameters='{}' name='reply_to_user' parameters='{}' role='assistant' content='hello world' -------- Done -------- [ToolCall(name='get_weather', parameters='{}'), ToolCall(name='add_reminder', parameters='{}'), ToolCall(name='reply_to_user', parameters='{}'), ExampleMessage(role='assistant', content='hello world')] ```⁩⁩
v
i wonder if the llm is giving you a ⁨`{}`⁩ as paramters, i think it might be! you might prefer: ⁨`parameters map`⁩ as clearly the model doesn't like a pure string as that type
the streaming part is def a bug fyi! we'll be fixing that
d
The parameters thing as a string was just an example, my real code has depeer structures. But yes looking forward to the fix of the streaming! 💪
2 Views